A typical method for compressing data involves the use of a dictionary data base which lists commonly occurring data and replaces this commonly occurring data with a coded “token” which effectively represents that data using a reduced number of data bits.
Whenever an item of data occurs repeatedly this data item is replaced by its equivalent “token” and accordingly that data item is stored in a compressed form.
When data is stored in the compressed form, by using a look-up table each token can be replaced by its equivalent data item so that the original data can be reformed.
The above conventional compression technique has a number of drawbacks. These drawbacks include the number of data bits which are required to represent a token can also be significant with the result that significant storage space is required to store each token. In addition searching a data base which includes tokens can be quite cumbersome because tokens need to be reconverted to their original data item before a search of each of the data items can be properly conducted.